{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "451e68ce",
   "metadata": {},
   "source": [
    "# 07 Stateful Computation\n",
    "\n",
    "Original Documentation: https://docs.jax.dev/en/latest/stateful-computations.html\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "aadf6cd5",
   "metadata": {},
   "outputs": [],
   "source": [
    "import jax\n",
    "import jax.numpy as jnp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05a138e6",
   "metadata": {},
   "source": [
    "JAX transformations (`jax.jit()`, `jax.vmap()`, `jax.grad()`) require the functions they wrap to be pure. In ML, state may exist in forms like model parameters, optimizer state, and stateful layers.\n",
    "\n",
    "## A simple example\n",
    "\n",
    "Consider a toy example of a `Counter` class:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "64d2f8d6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "2\n",
      "3\n"
     ]
    }
   ],
   "source": [
    "class Counter:\n",
    "    \"\"\"A simple counter.\"\"\"\n",
    "\n",
    "    def __init__(self):\n",
    "        self.n = 0\n",
    "\n",
    "    def count(self) -> int:\n",
    "        \"\"\"Increments the counter and returns the new value.\"\"\"\n",
    "        self.n += 1\n",
    "        return self.n\n",
    "\n",
    "    def reset(self):\n",
    "        \"\"\"Resets the counter to zero.\"\"\"\n",
    "        self.n = 0\n",
    "\n",
    "\n",
    "c = Counter()\n",
    "for _ in range(3):\n",
    "    print(c.count())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "855b7122",
   "metadata": {},
   "source": [
    "`self.n` is the state that is maintained and modified as a side-effect during `count()` calls.\n",
    "\n",
    "For the sake of example, assume we must JIT compile `count()` to improve performance.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f61f5380",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "1\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "c = Counter()\n",
    "fast_count = jax.jit(c.count)\n",
    "\n",
    "for _ in range(3):\n",
    "    print(fast_count())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6812e88",
   "metadata": {},
   "source": [
    "Since the side-effect of incrementing `self.n` is only executed once at trace time, subsequent `fast_count()` calls will not repeat the side effect since the XLA compiler will remove them.\n",
    "\n",
    "## The solution: explicit state\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3d3e06cb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "2\n",
      "3\n"
     ]
    }
   ],
   "source": [
    "CounterState = int\n",
    "\n",
    "\n",
    "class Counter:\n",
    "    \"\"\"A simple counter.\"\"\"\n",
    "\n",
    "    def count(self, n: CounterState) -> int:\n",
    "        return n + 1\n",
    "\n",
    "    def reset(self):\n",
    "        return 0\n",
    "\n",
    "\n",
    "c = Counter()\n",
    "state = c.reset()\n",
    "\n",
    "fast_count = jax.jit(c.count)\n",
    "\n",
    "for _ in range(3):\n",
    "    state = fast_count(state)\n",
    "    print(state)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28042d4b",
   "metadata": {},
   "source": [
    "Allow the user to manage the necessary state so that `count()` can be pure and therefore JIT compiled.\n",
    "\n",
    "## Simple example\n",
    "\n",
    "Consider an example of a linear regression via gradient descent. Here, the only state is model parameters:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e6cd0576",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import NamedTuple\n",
    "\n",
    "LEARNING_RATE = 0.005\n",
    "\n",
    "\n",
    "class Params(NamedTuple):\n",
    "    weights: jnp.ndarray\n",
    "    biases: jnp.ndarray\n",
    "\n",
    "\n",
    "def init(rng):\n",
    "    \"\"\"Initialize model weights and biases randomly\"\"\"\n",
    "    W_key, b_key = jax.random.split(rng, 2)\n",
    "    W = jax.random.normal(W_key, ())\n",
    "    b = jax.random.normal(b_key, ())\n",
    "    return Params(W, b)\n",
    "\n",
    "\n",
    "def loss(params, x, y):\n",
    "    \"\"\"Compute the MSE of the model's prediction on x against y\"\"\"\n",
    "    preds = params.weights * x + params.biases\n",
    "    return jnp.mean((preds - y) ** 2)\n",
    "\n",
    "\n",
    "@jax.jit\n",
    "def update(params, x, y):\n",
    "    loss_value, grads = jax.value_and_grad(loss)(params, x, y)\n",
    "\n",
    "    # Adjust weights by gradient direction amplified by learning rate\n",
    "    return loss_value, jax.tree.map(lambda param, g: param - g * LEARNING_RATE, params, grads)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec1c2281",
   "metadata": {},
   "source": [
    "Notice that we manually pipe the params in and out of the update function.\n",
    "\n",
    "We can write a simple train loop to train our regression model:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "f1c2db4a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAHHCAYAAACRAnNyAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjUsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvWftoOwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAShBJREFUeJzt3Qd4leX5x/E7OyQkATKAMMKekSFbRFCGIFVwK7TF0bqodVfRquD4a7Wl1lFw1FELoqi4QUEZsveUDSHsECB7J+d/3U84MQkBzgknec/4fq7r5cy8POfJSc4vz/Sz2Ww2AQAAcEP+VhcAAADgTAgqAADAbRFUAACA2yKoAAAAt0VQAQAAbougAgAA3BZBBQAAuC2CCgAAcFsEFQAA4LYIKoBFbrnlFmnRokW1vnbixIni5+fn8jLBMwwaNMgcgC8gqACVaABw5FiwYIH4asCqW7eueALdIeTDDz+USy65ROrVqydhYWFywQUXyDPPPCPZ2dniLpKSkhx+3+lzAV/ix14/QEX/+9//Ktz+73//K3PnzjUfeOUNHTpUGjZsWO3/p7CwUEpKSiQkJMTpry0qKjJHaGioWBFUPv30U8nKyhJ3VlxcLGPGjJFPPvlEBgwYINdcc40JKj///LNMnz5dOnXqJPPmzTuv76GraGiaNWtWhfv+8Y9/yIEDB+Sf//xnhfuvvvpqCQoKMteDg4NrtZyAFQgqwDn86U9/kjfeeMP8dX42OTk55oPQ23lKUHnhhRfk8ccfl4cfflhefvnlCo99/fXXMnr0aBk2bJjMnj27Vsvl6PvkN7/5jWzevJkWFPg8un6AatDxAYmJibJmzRrTraAfPPqhqL788ksZOXKkxMfHm9aS1q1by7PPPmv+wj/bGBV78//f//53eeutt8zX6df36tVLVq1adc4xKnpbQ9UXX3xhyqZf27lzZ5kzZ85p5dduq549e5oWGf1/3nzzTZePe5k5c6b06NFD6tSpIzExMfLb3/5WDh48WOE5R44ckVtvvVWaNm1qytu4cWMZNWpUhQ/n1atXy+WXX27Ooedq2bKl3HbbbWf9v3Nzc004adeunQkslV155ZUybtw4UzfLly8vCwatWrWq8nz9+vUz9VW55c3++ho0aCA33XST7N+/3+H3iSvHqOj3U7932no0adIkadKkiURERMh1110n6enpkp+fL/fff7/ExcWZbjutc72vMkdeE1DbAmv9fwS8xPHjx2XEiBHml7l+CNu7EN5//33zYfDggw+ay59++kmeeuopycjIOO0v+6pot0RmZqbceeed5sPnpZdeMt0We/bsKWvyP5PFixfL559/Lvfcc4/5oHr11Vfl2muvleTkZImOjjbPWbdunQwfPtyEAv1Q0wClYzZiY2NdVDOldaAfhhqyNCgcPXpU/vWvf8mSJUvM/6/jRZSWbcuWLXLvvfea0JaSkmK62bS89tva6qFle+yxx8zXaYjR13iuejh58qTcd999EhhY9a+53//+9/Lee+/JN998I3379pUbb7zR3KehUMttt2/fPhNmyn/vnn/+eXnyySflhhtukD/84Q9y7Ngxee2110wYKf/6zvY+qQla1xoytK527dplyqTvGX9/f1MfGkb1tej3RwOfvi+r85qAWqVdPwDObPz48drnU+G+gQMHmvumTp162vNzcnJOu+/OO++0hYWF2fLy8sruGzdunC0hIaHs9t69e805o6OjbSdOnCi7/8svvzT3f/3112X3Pf3006eVSW8HBwfbdu3aVXbfhg0bzP2vvfZa2X1XXnmlKcvBgwfL7tu5c6ctMDDwtHNWRcsdHh5+xscLCgpscXFxtsTERFtubm7Z/d988405/1NPPWVunzx50tx++eWXz3iuWbNmmeesWrXK5oxXXnnFfJ1+/ZloHetzrrnmGnM7PT3dFhISYnvooYcqPO+ll16y+fn52fbt22duJyUl2QICAmzPP/98hedt2rTJ1GH5+8/2PjmXkSNHVnh/lKfn1cNu/vz55v/ROtf6t7v55ptN2UeMGFHh6/v161fh3M68JqC20fUDVJN2VWirQWX6F62dtoykpqaawZw6NmHbtm3nPK/+ZV+/fv2y2/q1SltUzmXIkCGmK8euS5cuEhkZWfa12nqiA0h1fIZ2Tdm1adPG/NXvCtpVoy0h2qpTfrCvdod16NBBvv3227J60sGg2m2hf+1Xxf5XvLZ66OBjR2m9K21VOhP7Y9rSpbSetA60+6T8eKSPP/7YtLg0b97c3NbWHB0ErS0P+r21H40aNZK2bdvK/PnzHXqf1ARtESrf6tanTx/zWip3len92qWjA7Kr85qA2kRQAapJxwFUNetCuzJ0ZkZUVJT58NNuC23yVzpe4FzsH4h29tBypg/zs32t/evtX6sBQsdvaDCprKr7qkO7SlT79u1Pe0yDiv1x/QD/29/+ZgazaneIdjFoN5eOW7EbOHCg6R7SLiodo6LjV7S7pqrxFVWFEHtgcTTMaEjUD/Bly5aZ27t37zbjS/R+u507d5oPf/0A1+9t+WPr1q2mjh15n9SEyt9/fQ+qZs2anXa/BhP7+9HZ1wTUJsaoANVUvuXELi0tzXy4akDRcR/auqGtCmvXrpVHH33UfDicS0BAQJX3OzJB73y+1go6wFMHtuoA4O+//96MkdBxFjqup3v37maMjs4w0nEVOlNHn6OtAzp1V+8703ouHTt2NJcbN240rUdV0ceUTlO207LogFdtVbnooovMpY7vuP7668ueo99DLZcGrKrqu3KZqnqf1JQzff/P9b5w9jUBtYmgAriQdmPo4EltStcWAru9e/eKO9BZHxqcdKBlZVXdVx0JCQnmcvv27XLZZZdVeEzvsz9up2HuoYceMof+Zd+tWzcTRMqvZ6NdL3rogE8dbDx27FiZMWOGGfRZlYsvvth0G+lzn3jiiSo/fHV9HPtsH7vw8HBzW2csTZ482XT7aNdb+W4yLa9+wOtgVJ1V5A288TXBe9D1A7iQ/QOxfAtGQUGB/Pvf/xZ3KZ+OY9EWjEOHDlUIKa5aT0Sn8Wogmjp1aoUuGj2/diPoWBWlY3by8vJO+8DUrhj712mXVeXWIA0y6mzdP9oqouunaDDSoFKZjpPRmS867VkDUHnazaN1884778iGDRsqdPsonYGl9ajdUZXLprc1qHoab3xN8B60qAAupN0FOiZE1+j485//bJrTdUVbd+p60SmqP/zwg/Tv31/uvvtuM8D29ddfN+t9rF+/3qFz6MDW55577rT7de0NHUSrY090AKl2g918881l05N1yvEDDzxgnrtjxw4ZPHiwGcCp3S86jVhXZ9Xn6lRe9cEHH5iQp2N+NMTouJK3337bdK1dccUVZy2jTtHVabVaFh1zomNdtBtGpy5ra412D+n5K9PzaljSoKMf3vp15Wk59LVPmDDBTJXWriV9vraaafnvuOMO87WexBtfE7wHQQVwIV2rRGeoaDfGX//6VxNadCCtfiDrX+/uQBf00tYN/eDRMSE60FLH02hrhyOzkuytRPq1VX3gaVDRxey0VePFF180Y3O0S0XDhoYG+0we/X81xPz4448mzGlQ0cG2Oi7EHg406KxcudJ082iA0UGgvXv3lmnTppluirPRkKHn0i4ebR3R8mq5tYxPP/20+R5puSrTrrGrrrrK/B/a+qStQ1WFIO0i0eXttRXC/np0zRf9Wk/kja8J3oEl9AEY+le0zljScSIA4C4YowL4IJ2iXJ6Gk++++67CsuwA4A5oUQF8kC6fr90zureNrmsyZcoUMzhVx3ToWhoA4C4YowL4IN3r56OPPjKLq+nCa7rp3v/93/8RUgC4HVpUAACA22KMCgAAcFsEFQAA4LY8eoyK7k+hK0jqwkS6sBYAAHB/OupEF3DU7Sl0Py2vDSoaUirvCgoAADyD7lbetGlT9w4qBw8eNCtX6kqZuveHbjWv27jrfiHnYt+eXV+oLqntSrpEuC4zrqsyBgUFufTc3oa6cg715TjqynHUlXOoL2vrKiMjwzQ02D/H3Tao6IZjut/IpZdeaoJKbGysWXhKlx13hL27R0NKTQQVXQJcz8ub+OyoK+dQX46jrhxHXTmH+nKPunJk2IalQUX3/dBEpS0odufavwMAAPgOS2f9fPXVV6aL5/rrrzcbf3Xv3t3sjAoAAGB5i8qePXvM0t0PPvigPP7447Jq1Sr585//LMHBwTJu3LjTnq9LfOtRvo/L3iylhyvZz+fq83oj6so51JfjqCvHUVfOob6srStnzmXpyrQaSLRFZenSpWX3aVDRwLJs2bLTnj9x4sSy7cfLmz59uuk/AwAA7k8nz4wZM0bS09PPOcY00OqN0Tp16lThvo4dO8pnn31W5fMnTJhgWl8qjxrWkcg1MZh27ty5MnToUAZanQN15Rzqy3HUleOoK+dQX9bWlb1HxBGWBhWd8bN9+/YK9+3YsUMSEhKqfL5unqZHZVpxNfVGq8lzexvqyjnUl+OoK8dRV86hvqypK2fOY+lg2gceeECWL19udm3dtWuX6cJ56623ZPz48VYWCwAAuAlLg0qvXr1k1qxZZrv5xMREefbZZ+WVV16RsWPHWlksAADgJixfmfY3v/mNOQAAACpj92QAAOC2CCoAAMBtEVQAAIDbIqicwZGMPEnJtboUAAD4NoJKFd5bslcGvLxIvt1P9QAAYCU+iauQ2CTKXO5M95OSEst2GAAAwOcRVKrQtWk9qRPkL9lFfrIzJcvq4gAA4LMIKlUIDvSXngn1zfVle09YXRwAAHwWQeUM+rRsYC6X7yGoAABgFYLKGfRrVRpUViadlGLGqQAAYAmCyhl0ahwhdQJskplXJFsOpVtdHAAAfBJB5QwCA/yldWRpS8qy3cetLg4AAD6JoHIWbaNKg8pSggoAAJYgqJxF21MtKquSTkhhcYnVxQEAwOcQVM6icZhI/bAgySkolo0H0qwuDgAAPoegchb+fr9OU166i+4fAABqG0HlHPq2LF34jXEqAADUPoLKOfRtFW0u1ySflLzCYquLAwCATyGonEOrmDCJiwiRgqISWZt80uriAADgUwgq5+Dn5yf9Wpe2qiyn+wcAgFpFUHHARaeCCuNUAACoXQQVB/RrFWMu1+9Pk+z8IquLAwCAzyCoOKBZgzrSpF4dKSqxycokdlMGAKC2EFQcHKdycZvSVpUlO1OtLg4AAD6DoOKgi9uWBpWfCSoAANQagoqD+reJET8/ke1HMyUlI8/q4gAA4BMIKg5qEB4sifFR5vriXbSqAABQGwgq1ej+WUz3DwAAtYKg4oQBpwbU/rwrVWw2m9XFAQDA6xFUnNCjRX0JDfKXY5n5ZqwKAACoWQQVJ4QEBkiflqWr1NL9AwBAzSOoOGkA05QBAKg1BJVqDqhdsfe45BcVW10cAAC8GkHFSe0bRkhsRIjkFZbImqSTVhcHAACvRlA5j+X0dfYPAACoOQSV8xinwoBaAABqFkGlGuwtKpsPpcvJ7AKriwMAgNciqFRDXGSoGauia74t2U2rCgAANYWgcr67Ke8gqAAAUFMIKuc5TmXRzmMspw8AQA0hqFRT31bREhLoL4fT82TH0SyriwMAgFciqFRTaFCACStqwfYUq4sDAIBXIqich0HtY83lgu3HrC4KAABeiaByHga1jzOXq/edkKz8IquLAwCA1yGonIeWMeHSIjpMCottsoRVagEAcDmCiotaVej+AQDA9Qgq52ngqXEqC7enME0ZAAAXI6icp36npikfSs+TnSlMUwYAwJUIKueJacoAANQcgooLME0ZAAAvDCoTJ04UPz+/CkeHDh3EUwfUrkpimjIAAF7VotK5c2c5fPhw2bF48WLxxGnKCUxTBgDA+4JKYGCgNGrUqOyIiSnd7M/TDGpH9w8AAK4WKBbbuXOnxMfHS2hoqPTr109eeOEFad68eZXPzc/PN4ddRkaGuSwsLDSHK9nP5+h5L27TQD5Yts8MqC0oKDDdWL7C2bryddSX46grx1FXzqG+rK0rZ87lZ7Nw8Y/Zs2dLVlaWtG/f3nT7TJo0SQ4ePCibN2+WiIiIKse06HMqmz59uoSFhYmVCopFJqwKkCKbnzzWtUgaW1scAADcVk5OjowZM0bS09MlMjLSfYNKZWlpaZKQkCCTJ0+W22+/3aEWlWbNmklqauo5X2h10t7cuXNl6NChEhQU5NDX3P7fNbJo53F5ZFhbuWNAS/EV1akrX0Z9OY66chx15Rzqy9q60s9vHerhSFCxvOunvHr16km7du1k165dVT4eEhJijsq04mrqjebMuYd0amSCyoIdqTL+snbia2ry++CNqC/HUVeOo66cQ31ZU1fOnMfywbTlaTfQ7t27pXHjxuKJBndsaC7X7DspJ7ILrC4OAAAez9Kg8vDDD8vChQslKSlJli5dKldffbUEBATIzTffLJ6oSb060rFxpJTYROZvY5VaAAA8OqgcOHDAhBIdTHvDDTdIdHS0LF++XGJjS6f6eqIhHUsXf/tx21GriwIAgMezdIzKjBkzxNto989rP+2SRTtSpaCoRIID3ap3DQAAj8KnqIt1aRIlsREhZin9FXuPW10cAAA8GkHFxfz9/eSyU3v//LiVcSoAAJwPgkoNGHxqnMq8rUfFjZapAQDA4xBUasDFbWPM2JQDJ3Nlx9Esq4sDAIDHIqjUgLDgQLm4TUxZqwoAAKgegkoNd//8SFABAKDaCCo1ZHCH0lVq1+1Pk9SsX/cnAgAAjiOo1JBGUaGS2CRSdCwtq9QCAFA9BJVaaFVhnAoAANVDUKlBQzuVBhVdpTavsNjq4gAA4HEIKjWoc3yk2agwt7BYft6ZanVxAADwOASVGuTn5yfDOpe2qszZfMTq4gAA4HEIKjXs8s6NynZTLiousbo4AAB4FIJKDevVooFEhwdLWk6hrNx7wuriAADgUQgqNSzA30+GdDzV/bOF7h8AAJxBUKkFlyeWBpUfthyVkhI2KQQAwFEElVpwUesYqRsSKEcy8mTjwXSriwMAgMcgqNSC0KAAGdQ+1lxn9g8AAI4jqNTy7J8fthwRm66rDwAAzomgUksu7RAnwQH+sic1W3alZFldHAAAPAJBpZboGJWL28aY63T/AADgGIJKLbr81Cq13/9CUAEAwBEElVqk66n4+4lsPpgh+0/kWF0cAADcHkGlFkXXDTEr1arvWfwNAIBzIqjUspFdGpvLbzcdtrooAAC4PYJKLRue2Ej8/ETWJafJgZN0/wAAcDYElVoWFxEqfVqWdv/M3kT3DwAAZ0NQscDIC0q7f76h+wcAgLMiqFjg8sRGZvbPhv1pzP4BAOAsCCqWdf9Em+vf0aoCAMAZEVQsnv1DUAEA4MwIKhbO/jHdPwfS6f4BAOAMCCoWiakbIv1al3b/sKYKAABVI6hY6IpTs3++3UhQAQCgKgQVCw3vXNr9s+lguuw7nm11cQAAcDsEFYv3/rmodYy5TvcPAACnI6i4yeyfbzYQVAAAqIyg4gbdP4H+fvLL4QzZlZJldXEAAHArBBWL1Q8PlkvaxZrrX64/aHVxAABwKwQVNzCqW7y5/HL9IbHZbFYXBwAAt0FQcQNDOzWUsOAAST6RI+v2p1ldHAAA3AZBxQ2EBQfKsE4NzfWv1h+yujgAALgNgoqbGNWtibn8ZuMhKSousbo4AAC4BYKKm7i4bYw0CA+W1KwCWbL7uNXFAQDALRBU3ERQgL+MPLWk/pfrmP0DAIAiqLiR0d1LZ/98v+WI5BYUW10cAAAsR1BxIxc2ry9N69eR7IJimbf1qNXFAQDAcgQVN+Ln51duTRW6fwAAIKi4mdGnZv8s2H5MTmYXWF0cAAAs5TZB5cUXXzQtCvfff7/4srYNI6Rj40gpKrGxozIAwOe5RVBZtWqVvPnmm9KlSxeri+IWrule2qry2doDVhcFAADfDipZWVkyduxYefvtt6V+/fpWF8ctjOoeLwH+frIuOU12H2NHZQCA7wq0ugDjx4+XkSNHypAhQ+S5554763Pz8/PNYZeRkWEuCwsLzeFK9vO5+ryOqB8aIAPaRMuCHakyc1WyPDS0rbgzK+vKE1FfjqOuHEddOYf6sraunDmXn83C7XpnzJghzz//vOn6CQ0NlUGDBkm3bt3klVdeqfL5EydOlEmTJp12//Tp0yUsLEy8yfrjfvLejgCJCrbJxAuLxd/P6hIBAOAaOTk5MmbMGElPT5fIyEj3DCr79++Xnj17yty5c8vGppwrqFTVotKsWTNJTU095wutTtrTsg0dOlSCgoKktuUXlUj/lxZIem6RvDeuh1zcJlrcldV15WmoL8dRV46jrpxDfVlbV/r5HRMT41BQsazrZ82aNZKSkiIXXnhh2X3FxcWyaNEief31100gCQgIqPA1ISEh5qhMK66m3mg1ee6z/78iV3VtIh8u3ydfbDgsl3ZsJO7OqrryVNSX46grx1FXzqG+rKkrZ85jWVAZPHiwbNq0qcJ9t956q3To0EEeffTR00KKL7q2R1MTVHRJ/Yy8QokM5YcJAOBbLAsqERERkpiYWOG+8PBwiY6OPu1+X9W1aZS0iasru1Ky5LuNh+Wm3s2tLhIAAL41PRlnpgvgXXthU3OdNVUAAL7I8unJ5S1YsMDqIridq7s3kZe/3yarkk5KUmq2tIgJt7pIAADUGlpU3FyjqFAZ0DbWXP+cVhUAgI8hqHjIoFr12dqDUlxi2bI3AADUOoKKBxjWqaFE1QmSg2m5snhXqtXFAQCg1hBUPEBoUIAZq6JmrEy2ujgAANQagoqHuKl3M3M595ejcizz19V5AQDwZgQVD9GhUaR0a1ZPikpsTFUGAPgMgooHualXaavKx6v2i4V7SQIAUGsIKh7kyq7xEh4cIHtTs2X5nhNWFwcAgBpHUPEg4SGBclW3eHN9xioG1QIAvB9BxcPc1Kt0v5/Zm49IWk6B1cUBAKBGEVQ8TJemUdKxcaQUFJXIrHUHrS4OAAA1iqDigRsV2gfVzljJoFoAgHcjqHig0d2aSEigv2w/minr9qdZXRwAAGoMQcUDRYUFycgLGpvr05YzqBYA4L0IKh7qt/0SzOXXGw/JyWwG1QIAvBNBxUN1b1ZPOseXDqr9ZPV+q4sDAECNIKh48KDa3/UtbVX534p9UlLCoFoAgPchqHiwUd2aSERooOw/kSsLdx6zujgAALgcQcWD1QkOkOt7lE5V/nDZPquLAwCAyxFUPNxv+5auVDt/e4rsP5FjdXEAAHApgoqHaxVbVwa0jRFd923aCqYqAwC8C0HFC/z21KDaj1clS15hsdXFAQDAZQgqXmBwhziJjwqVkzmF8t2mw1YXBwAAlyGoeIHAAH8Z06d0rMp/GVQLAPAiBBUvcWOv5hIU4Cfr96fJuuSTVhcHAACXIKh4idiIELmya7y5/t6SJKuLAwCASxBUvMht/VuaSx2ncjg91+riAABw3ggqXiSxSZT0adlAikpsjFUBAHgFgoqXue3i0laV6SuSJbeAqcoAAM9GUPEyQzo2lOYNwiQ9t1A+X3fA6uIAAHBeCCpeJsDfT265qIW5/u7iveyqDADwaAQVL3R9z6ZSNyRQdh/LlkXsqgwA8GAEFS8UERokN/Yq3VX5XaYqAwA8GEHFS2n3j7+fyKIdx2Tn0UyriwMAQLUQVLxUswZhMrRTQ3P9nZ/3Wl0cAACqhaDixe64pJW5nLXuoKRk5FldHAAAnEZQ8WI9EhpIz4T6UlBcwlgVAIBHIqh4ubsGtjaX05bvk4y8QquLAwCAUwgqXu6yDnHSNq6uZOYXyUcrkq0uDgAATiGoeDl/f7+ysSr/WbxX8otYVh8A4OVBZf/+/XLgwK/Ls69cuVLuv/9+eeutt1xZNrjIqG5NpGFkiKRk5suX6w5ZXRwAAGo2qIwZM0bmz59vrh85ckSGDh1qwsoTTzwhzzzzTHVOiRoUHOgvt5/arHDqot0sqw8A8O6gsnnzZundu7e5/sknn0hiYqIsXbpUpk2bJu+//76rywgXuLl3c4kIDZQ9x7Jl3tajVhcHAICaCyqFhYUSEhJirs+bN0+uuuoqc71Dhw5y+PDh6pwStbCs/m/7JpjrUxfuFpuNVhUAgJcGlc6dO8vUqVPl559/lrlz58rw4cPN/YcOHZLo6GhXlxEucmv/FqYbaG1ymizbc9zq4gAAUDNB5W9/+5u8+eabMmjQILn55pula9eu5v6vvvqqrEsI7icuIlRu7Fm6WeFrP+6yujgAAJxToFSDBpTU1FTJyMiQ+vXrl91/xx13SFhYWHVOiVpy16DWMmNVsmlRWZ10Qnq2aGB1kQAAcG2LSm5uruTn55eFlH379skrr7wi27dvl7i4uOqcErWkSb06cu2FTc31V3+iVQUA4IVBZdSoUfLf//7XXE9LS5M+ffrIP/7xDxk9erRMmTLF1WWEi90zqI0E+PvJoh3HZP3+NKuLAwCAa4PK2rVrZcCAAeb6p59+Kg0bNjStKhpeXn31VYfPo6GmS5cuEhkZaY5+/frJ7Nmzq1MkOKF5dJiM6hZvrr/+006riwMAgGuDSk5OjkRERJjrP/zwg1xzzTXi7+8vffv2NYHFUU2bNpUXX3xR1qxZI6tXr5bLLrvMtNZs2bKlOsWCE8Zf2kb8/ETmbU2RLYfSrS4OAACuCypt2rSRL774wiyl//3338uwYcPM/SkpKaZlxFFXXnmlXHHFFdK2bVtp166dPP/881K3bl1Zvnx5dYoFJ7SOrSu/6WJvVWGsCgDAi2b9PPXUU2YZ/QceeMC0gmiXjb11pXv37tUqSHFxscycOVOys7PLzleZDuDVw05nHdkXoNPDleznc/V53cldAxLk6w2HZPbmI/LLgZPStmHdap3HF+rKlagvx1FXjqOunEN9WVtXzpzLz1bNJUp1jx9dhVbXUNFuH6X7/WiLiq5Q66hNmzaZYJKXl2daU6ZPn25aWaoyceJEmTRp0mn369cwLbp6/rPdXzae8Jfu0SVyS7sSq4sDAPABOTk5psEjPT39nD0x1Q4qdvZdlHW8SXUUFBRIcnKyKawOzH3nnXdk4cKF0qlTJ4daVJo1a2bWdHGmy8nRtKer7uqGi0FBQeKtth3JlCvfWGaufz2+n3RoVDr2yBm+UleuQn05jrpyHHXlHOrL2rrSz++YmBiHgkq1un5KSkrkueeeM1OSs7KyzH06uPahhx4yOyjbW1gcERwcbMa8qB49esiqVavkX//6l1n5tjLdX8i+x1B5WnE19UaryXO7gwuaNZDfdGks32w8LK/O3yNv/75ntc/l7XXlatSX46grx1FXzqG+rKkrZ85TraCiYeQ///mPmbHTv39/c9/ixYtN14x24eig2OrSEFS+1QQ17/4h7eS7TYdl7i9HZcP+NOnarJ7VRQIAoPpB5YMPPjBdNPZdk5Wuh9KkSRO55557HA4qEyZMkBEjRkjz5s0lMzPTjDVZsGCBmUmE2tMmrq5c3b2pfLb2gEyeu0M+uI39mgAAHjw9+cSJE1UOmNX79DFH6XTm3//+99K+fXsZPHiw6fbRkKL9YKhd9w1uK4H+frJwxzFZleT49xAAALcLKjrT5/XXXz/tfr1PW1Ycpd1HSUlJpqtHQ8u8efMIKRauVnv9qZ2V//79djnPMdYAAFjX9fPSSy/JyJEjTbCwr3mybNkyswDcd99955qSodbde1kb+WzNAVmx94Qs3X1c+reJsbpIAAAfV60WlYEDB8qOHTvk6quvNpsS6qHL6OvS9x9++KHrS4laEV+vjozp09xc//sPtKoAADy0RUXFx8efNmh2w4YNpjvnrbfeckXZYIF7Lm0tH6/aL+uS0+SHX47K5Z0bWV0kAIAPq1aLCrxXXESo3H5xS3P9pTnbpKiY1WoBANYhqOA0dw5sJQ3Cg2X3sWz5ZHXpysMAAFiBoILTRIQGmYG16p/zdkhOQZHVRQIA+CinxqjogNmz0UG18A5j+yTIe0uSJPlEjvzn571y7+C2VhcJAOCDnGpRiYqKOuuRkJBgFnCD5wsO9JeHL29vrr+5aI+kZrGtAQDAzVtU3nvvvZorCdzOby5oLG8v2iObDqbLaz/ulEmjEq0uEgDAxzBGBWfk7+8nE64o3Sph2opkSUrNtrpIAAAfQ1DBWV3UOkYGtY+VohKb/G3ONquLAwDwMQQVnNOEER3F309k9uYjsnzPcauLAwDwIQQVnFP7RhFlS+tP+voXKS5haX0AQO0gqMAhDw5tL5GhgbL1cIbMXL3f6uIAAHwEQQUO0ZVq7xvSrmzDwsy8QquLBADwAQQVOOz3/RKkVWy4pGYVyOs/7bK6OAAAH0BQgcOCAvzlryM7muvvLtnLdGUAQI0jqMApl7aPk0vaxUphsU2e/26r1cUBAHg5ggqc4ufnJ0+O7CgB/n4y95ejsmB7itVFAgB4MYIKnNa2YYSM69fCXJ/41RbJLyy2ukgAAC9FUEG1PDC0rcRFhEjS8Rx5e3GS1cUBAHgpggqqJSI0SJ44NbB26qK9cjzP6hIBALwRQQXVdlXXeOnXKlryi0rk8yTeSgAA1+PTBec1sPbZ0Z0l0N9PNp/0lx+3MbAWAOBaBBWclzZxEXLrRQnm+nPfbpPcAgbWAgBch6CC8zZ+UCupF2yTA2l58sZ8VqwFALgOQQXnLTwkUK5pUWKuT124W7YdybC6SAAAL0FQgUt0aWCTIR1ipajEJo99tkmKS2xWFwkA4AUIKnAJPz+Rp6/sKBEhgbJ+f5p8uIy1VQAA54+gApdpFBkqfxnRwVx/6fvtcjAt1+oiAQA8HEEFLjW2d3PpmVBfcgqK5a+zNonNRhcQAKD6CCpwKX9/P3nx2gskOMBf5m8/Jl9vPGx1kQAAHoygghpZW2X8pW3M9UlfbZET2QVWFwkA4KEIKqgRdw9qLe0a1pXj2QXy1JebrS4OAMBDEVRQI4ID/eXv13eVAH8/+WbjYfmWLiAAQDUQVFBjujStJ/cMam2uP/nlZknNyre6SAAAD0NQQY2697K20qFRhBmn8gSzgAAATiKooMa7gP5xQ1ezw/L3W47KVxsOWV0kAIAHIaigxnWOj5I/D25rrj/15RZJycizukgAAA9BUEGtzQK6oEmUpOcWyqOfbaQLCADgEIIKakVQQGkXkHYF6UJwHy7fZ3WRAAAegKCCWtOuYYQ8fmovoOe/3So7jmZaXSQAgJsjqKBWjbuohQxsFyv5RSXy54/WSX5RsdVFAgC4MYIKapWfn5+8fH0XiQ4Plm1HMuXlOdutLhIAwI0RVFDr4iJC5aXrupjr7yzeKz/vPGZ1kQAAboqgAksM7thQftc3wVx/6JMNcpxVawEAVSCowDKPX9FR2sTVlZTMfHngkw1SUsKUZQBARQQVWKZOcIC8MeZCCQ3yl0U7jsmUhbutLhIAwM0QVGCp9o0i5JmrEs31f/ywXVbuPWF1kQAAboSgAstd37OpXNO9iWjPz70frWW8CgDAPYLKCy+8IL169ZKIiAiJi4uT0aNHy/btTFf1xSnLz45OlNax4XI0g/EqAAA3CSoLFy6U8ePHy/Lly2Xu3LlSWFgow4YNk+zsbCuLBQuEhwTKv8f2KBuv8u8Fu6wuEgDADQRa+Z/PmTOnwu3333/ftKysWbNGLrnkEsvKBWvHq/zls43yj7k7pEvTenJJu1iriwUA8NWgUll6erq5bNCgQZWP5+fnm8MuIyPDXGpLjB6uZD+fq8/rjVxZV6O7NpRVSU1k5pqDZrzKrLv7SrP6YeJNeG85jrpyHHXlHOrL2rpy5lx+NpvNLQYDlJSUyFVXXSVpaWmyePHiKp8zceJEmTRp0mn3T58+XcLCvOvDzJcVloi8ujlAkrP9pEmYTe5PLJbgAKtLBQBwlZycHBkzZoxpoIiMjPSMoHL33XfL7NmzTUhp2rSpwy0qzZo1k9TU1HO+0OqkPR03M3ToUAkKCnLpub1NTdTV4fQ8GT1lmZzILpSrujSWv1+XaAbdegPeW46jrhxHXTmH+rK2rvTzOyYmxqGg4hZdP3/605/km2++kUWLFp0xpKiQkBBzVKYVV1NvtJo8t7dxZV01jwkyg2vHvrNCvtp4WLon1Jdb+7cUb8J7y3HUleOoK+dQX9bUlTPnsXTWjzbmaEiZNWuW/PTTT9KypXd9EOH89G0VbZbZV899u1WW7k61ukgAgFpmaVDRqcn/+9//zBgTXUvlyJEj5sjNzbWyWHAjt/VvIaO7xUtxiU3u/t9aSUpl6joA+BJLg8qUKVNM/9SgQYOkcePGZcfHH39sZbHgRnRcyovXdpFuzepJem6h3P7BKnMJAPANlnf9VHXccsstVhYLbiY0KEDe+n0PiY8Kld3HsuVP09dKUXGJ1cUCANQC9vqBR4iLCJW3x/WUOkEB8vPOVDNmBQDg/Qgq8Bid46Pknzd2M9ffX5okHy7fZ3WRAAA1jKACjzI8sZE8cnl7c/3pLzfLj1uPWl0kAEANIqjA49wzqLXc0LOp6AbLf5q+TjbsT7O6SACAGkJQgUfOBHr+6gvMhoW5hcVmJlDy8RyriwUAqAEEFXikoAB/+ffYC6VzfKSkZhXIuPdWyonsAquLBQBwMYIKPFbdkEB575Ze0qReHdmbmi1/+GCV5BYUW10sAIALEVTg0eIiQ+WD23pJZGigrE1Ok3umrZFC1lgBAK9BUIHHaxMXIe/e0ktCg/xl/vZj8tAnG6RER9oCADweQQVeoWeLBjLltz0k0N9PvtpwSJ7+aotZ5RgA4NkIKvAal7aPk8k3dhM/PzGLwU2eu8PqIgEAzhNBBV7lqq7x8uyoRHP9tZ92yduL9lhdJADAeSCowOv8tm9C2eq1z3+3VT5YmmR1kQAA1URQgdeuXjv+0tbmuo5XmbaCfYEAwBMRVOC1q9c+PKy93HlJK3P7iVmb5eNVyVYXCwDgJIIKvDqsPDaig9zWv6W5/djnm+TTNQesLhYAwAkEFXh9WHnyNx1lXL8E0dnKj3y6gbACAB6EoAKfCCsTr+osY/s0N2Hl4ZkbGLMCAB6CoAKfCSvPjU6UWy5qUTZm5d3Fe60uFgDgHAgq8Kmw8vSVneTOgaUDbJ/55hf594JdVhcLAHAWBBX43gDb4R3k/iFtze2X5myXyT9sZ7l9AHBTBBX4ZFi5f0g7eXR4B3P71Z92yV+/2CzFbGQIAG6HoAKfdfeg1vLs6ESzN9C0FckyftpaySsstrpYAIByCCrwab/rmyBvjLlQggP8Zc6WIzLu3ZWSkVdodbEAAKcQVODzrrigsbx/Wy+pGxIoK/aekBvfXC4pGXlWFwsAQFABSl3UOkZm3NFXYuqGyNbDGXLt1KWyNzXb6mIBgM8jqACnJDaJks/u7icJ0WGy/0SuXPPvJbIq6YTVxQIAn0ZQAcpJiA6XT++6SLo0jZKTOYUy9u0V8hlL7gOAZQgqQCWxESHy8R39ZERiIykoLpGHZm6Ql7/fJiVMXwaAWkdQAapQJzjAzAYaf2lrc/uN+btl/PS1klvA9GUAqE0EFeAM/P395JHLO8g/ru8qQQF+MnvzEbnxrWXMCAKAWkRQAc7h2h5NZdof+kr9sCDZeCBdrnx9sazZd9LqYgGATyCoAA7o3bKBfDG+v7SJqytHM/LlpreWyYfL97FHEADUMIIK4MSMIA0rOsi2sNgmT36xWR75dCPL7gNADSKoAE7Q1Wv/PfZCeWxEB/H3E/l0zQG5bupS2X8ix+qiAYBXIqgA1dh9+a6BreXD2/tIg/Bg2Xwww4xbWbjjmNVFAwCvQ1ABqql/mxj5+t6LpWvTKEnLKZRb3lspf5uzTQqLS6wuGgB4DYIKcB6a1KsjH9/ZT8b2aS46rnbKgt1yw5vL6AoCABchqADnKTQoQJ6/+gIzdiUiNFDWJafJFa/+LN9uPGx10QDA4xFUABe54oLG8t2fB8iFzetJZl6RWcl2wucbWc0WAM4DQQVwoWYNwkxXkC697+cn8tHK/Wag7cYDaVYXDQA8EkEFcLGgAH+z9P7/bu9jNjjclZIlV/97qUyeu4OBtgDgJIIKUIOzgn64/xIZ2aWxFJfY5NUfd8roN5bIjqOZVhcNADwGQQWoQfXDg80uzK/d3F3qhQXJlkMZMnrKcvnxoJ8JLwCAsyOoALXgyq7xpnXlsg5xZvn9r5ID5OZ3VtK6AgDnQFABaklcZKj8Z1xP+b/RnSUkwCbr9qfLyFd/lsk/bGe/IAA4A4IKUMvL71/fo4lM6Fosl7WPNa0rr/60y6y7smLPcauLBwBuh6ACWKB+iMjUsd3MInE6M2jPsWy58a3l8thnGyU9p9Dq4gGA2yCoABa2rugicfMeGCg3925u7puxar8MnrxAZq7eLyUMtgUAa4PKokWL5Morr5T4+HjzS/uLL76wsjiAJaLCguSFay6QT+7sJ61jwyU1q0Ae+XSjXDt1KQvFAfB5lgaV7Oxs6dq1q7zxxhtWFgNwC71bNpDZ910iE0Z0kPDgALNn0Kg3lphl+I9n5VtdPACwRKBYaMSIEeYAUCo40F/uHNhaRndvIi/O3iaz1h00y/DrBocPDm0nY/smmJVvAcBXWBpUnJWfn28Ou4yMDHNZWFhoDleyn8/V5/VG1JXr66tBnQB56ZrOckOPeHnmm22y9UimTPz6F3l/aZI8PLStDOsUZ7pLvR3vLcdRV86hvqytK2fO5Wez2dxixJ7+0p01a5aMHj36jM+ZOHGiTJo06bT7p0+fLmFhYTVcQsAaOqZ26VE/mX3AX7IKS8NJywibjEoolpYRVpcOAJyXk5MjY8aMkfT0dImMjPSeoFJVi0qzZs0kNTX1nC+0Omlv7ty5MnToUAkKCnLpub0NdVU79ZWVXyRv/5wk7y5NkrzC0s0NL+8UJw8PaystosPFG/Hechx15Rzqy9q60s/vmJgYh4KKR3X9hISEmKMyrbiaeqPV5Lm9DXVVs/VVPyhI/jKio/z+opbyz7k7ZOaa/fL9Lykyb9sxue7CpnLv4DbStL53tizy3nIcdeUc6suaunLmPIzKAzxMo6hQ+dt1XeS7+wbIpe1jzeaGH6/eL5f+fYE8+cVmOZKeZ3URAcBlLA0qWVlZsn79enOovXv3muvJyclWFgvwCB0aRcp7t/aWT+/qJxe1jjbL8X+4fJ9c8vJ8eebrX+RYJlOaAXg+S4PK6tWrpXv37uZQDz74oLn+1FNPWVkswKP0bNFApv+xr3z0x77Sq0V9KSgqkXeX7JUBL/0kE7/aIofScq0uIgBUm6VjVAYNGiRuMpYX8Hj9WkfLJ636yc87U2Xy3B2yfn+amc48bcU+ubp7E7lrYGtpFVvX6mICgFM8ajAtgHPPnrukXawMaBsjS3Ydlzfm75Jle47LJ6sPyMw1B8zeQncPbC2JTaKsLioAOISgAnhpYLm4bYw51uw7KVMW7JJ5W1PMCrd69GsVLbdf3FIu6xAn/v7ev3AcAM9FUAG8XI+E+vLOuF6y9XCGTFmwW77ddNi0sujRIjpMbu3fUq7r0VTCQ/h1AMD9MD0Z8BEdG0fKqzd3l0V/uVTuHNhKIkMDJel4jjz91Rbp+8KP8n/fbZUDJ3OsLiYAVEBQAXxMk3p1ZMKIjrJswmB5dlRnaRkTLpl5RfLWoj1yyUvz5Y7/rpaFO45Jia7dDwAWo60X8FHa1fO7fi1kbJ8EWbAjRf6zeK8ZgPvDL0fN0bR+Hbm5d3O5vmdTiYsItbq4AHwUQQXwcTqY9rIODc2x42imTF+RLJ+vPSAHTubKy99vN8v1D+3UUMb0aS79W8cw+BZArSKoACjTrmGETLyqszw6vIMZdDt9xT5Zm5wmszcfMUfzBmFyzYVN5JruTaV5tHfuKwTAvRBUAJymTnCAmQmkx7YjGfKRtrKsOyjJJ3LklXk7zdG7RQMTWq7o0lgiQ9nUDUDNIKgAOOeeQpNGJcqjIzrID1uOymdrD8jiXamyMumEOZ76aovpGrr2wiYyoG2sBAUwRh+A6xBUADgkLDhQRndvYg7dofnL9QdNaNlxNKtsIbn6YUEyPLGx/KZLY+nTsoEEEloAnCeCCgCnNYoKlTsHtpY7LmklWw5lyOdrD8pXGw5KalaBfLQy2RzR4cEyPLGRjDShJVoCGIQLoBoIKgDOa6l+3TdIj8ev6CAr9p6QbzYeljmbD8vx7AKZtiLZHDF1S0PLsE6NpG+raAkOpKUFgGMIKgBcQrt5+reJMcczozrL8j3HTXfQnC1HTEvL/5Ynm6NuSKAMbB8rwzo1lEHt4iQqjIG4AM6MoALA5XRArQ6s1ePZ0YmydPdx08oy95cUSc3KLxvTEujvJ71bNjCDcQd3aMiUZwCnIagAqPHQMrBdrDmeH22TDQfSZO4vR82xMyXLhBg9Jn39i9kkUZ93SbvY0i4ieogAn0dQAVBrdFXb7s3rm+MvwztIUmq2zNtaGlrW7DtpNklMWrZPPli2T4ID/KVHQj2JLfKTVkcyJbFpfTMmBoBvIagAsEyLmHD5w4BW5sjMKzQtK4t2HDObIuoS/sv2nBCRAPnqjWUSFxEiF7eJkb6to6Vfq2izFxHBBfB+BBUAbiEiNEgu79zIHDabTfamZsv8bUflsyW/yN7sQEnJzDer4+qh4qNCTfeQ/WjWgOACeCOCCgC3o4GjVWxdaVYvRGJObJbBwy6TDQczZdnu42Y2kY5zOZSeV2Vw0cG5FybUlzaxddlAEfACBBUAbi8k8NepzyqnoEjW7kszoeVMwSUyNFC6Na8vPZrXlwsT6km3ZvVMqw0Az0JQAeCRy/lf3DbGHJWDy+p9J2TD/nTJyCsy4130UNor1L5hhGlt6d6snnRpWk9ax4azzD/g5ggqALwuuBQVl8i2I5lmJtHa5JPmUgfn6n16TF+RbJ4XGuQvneOj5IImpUeXplGmy4nl/gH3QVAB4HW0lcS+tP+4i1qY+1Iy8spCy4YD6bLlYLpkFxSb23rYhQUHSOf4SBNgOjWOlA6NI6RtXITUCQ6w8BUBvougAsAnxEWGmp2d9VAlJTbZk5otmw6myaYDGeZSN1jMKSiWVUknzWGnDSw6lbpjo0jp0ChCOmiAaRTBFGmgFhBUAPgknRHUJq6uOa7uXnpfsYaXY1my8UC6bD2cIVuPZMjWw5lyIrtA9hzLNse3mw6XnSMiJFDaN4qQtg3rSuvYutJazxdbV5rUq8OMI8BFCCoAcIqOTWnbUINHRNl9uqbLsax82XZYx7dkmMutRzJlV0qmZOYXyep9J81Rno59aRlTGoJ0wK49ELWIDpfQILqQAGcQVADgLLRrJy4i1By6B5FdYXGJaWHR8LI7JUt2HcuSXSlZkpSaI3mFJaUtMoczKp1LpHFkqNl8UUOL/TIhOkwSosPNztIAKuKnAgCqudmidvvoUZ7OONp/MrcsvJQPMZl5RWa9Fz2Wm+0BKoqpGyzNG/waYjTANKkXJvH1QqVRZChTqeGTCCoA4EIaJlrGhJtjiDSs0IV0PLtA9h3PkX3Hs3+9PKGXOWYcTGpW6bE2Oa3KbikNKzr+RYNLk/p1ykKMDuqNDefXObwT72wAqKUupJi6IebokVD/tMcz8gol2YSXHEk6nm2uJ5/IkUPpuXIoLVcKi21yMC3XHGcSHhggU/YslYZRdaRhZIgJNjrbSS8b6hEVItHhIawTA49CUAEANxAZGlS29ktlOpVaB/TqonUaWkxgqXRdB/ZmF/nJtqNZ5jgTDSmxdUOkYVSoNIwIkUZRoeZ2tAlRwRUuw4MDmH4NyxFUAMDN6VRn0yISGVpla4w6npEjn3wzV9p36y2p2UVyNCNPjmbmyZH0fEkxl3mSmpVvpmAfycgzx7no7CVtgdHgoi1B0WVB5lSYCQ+RBuHBUi8sSOqHBZvnE2zgagQVAPACkXWCJD5cZEDbGAkKqnrzRR3oq2NgTIg5dWhgSc0skOPZ+afGyOhlvpm5pMe5upvKCw70l/phQVKvTrBEmfBSer1eeOmluW2OX8NNVJ0gpmzjrAgqAOBDA321q0ePc9GNHjXApGbny/FTAea4CTH266WXJ3MKJS2nQIpKbFJQVCJHM/LN4YzgAH+JCA08dQSVXdfusNNvV3yO/Tphx3sRVAAAVW702Dxaj7BzPldnNOm+SRpY0kxwKZSTej23UNKyT12eCjR6XR9L19u5haYrqqC4xMyI0qO6NOyEhwSYcp92GRwgYSGnLoMDzXo1umTNzmN+Erw1RSLDQsweT+EhgaWXwYFmb6eQQLqy3AFBBQBwXvTDXD/89Wha9RCaMwYcHQSs68tk5hWay4zcwrLbGXkVH/v1stz1/CJzLg07BTklpoXHcQHy4a71Z3ldIqGBAWbsTZ0gvQyQkKAAqRPkb67rofeHlHu8/HN/PSo+HhIYYLrJNFyZS/sRoI8RjiojqAAALKEfyNqdo4dInWqdQ1tkskzYKZTcgmLTspOTX2Tu0w0mswuKJCf/1KXePnV/Vl6h7Dt0RMIi60tuQUmFx/OLSsy5bTaR3MJic5wUZwLQ+QkK8DtDiAkou62BpsJzKj0/JMDfdPXpwoR6Pr0MPHVZdtu/4mPBZV9jf7z0MiTAJlYiqAAAPJZOt9YBuXo4o7CwUL777ju54oo+pw0+1kHHOYXFkldYLPmFJSao6PW8Cteruq+kwmO55R7PL3+9qNiM5zFHcYlZI6dC2YptUlhcGrrcwcjERjKs4gLMtYqgAgBAOdqqEKmHaempebpOjum60sMeYE6FmIKiEtPCY7+tgafC86p4jv2xohK9tJnLouLS/0NDWGkQ0sdLL/V26f1VPVZiurasRFABAMDidXJC/UvHsLijQtP6lGzZ/88OVwAAwG0RVAAAgNsiqAAAALdFUAEAAG6LoAIAANwWQQUAALgtggoAAHBbBBUAAOC23CKovPHGG9KiRQsJDQ2VPn36yMqVK60uEgAAcAOWB5WPP/5YHnzwQXn66adl7dq10rVrV7n88sslJSXF6qIBAABfDyqTJ0+WP/7xj3LrrbdKp06dZOrUqRIWFibvvvuu1UUDAAAWs3Svn4KCAlmzZo1MmDCh7D5/f38ZMmSILFu27LTn5+fnm8MuIyOjbB8CPVzJfj5Xn9cbUVfOob4cR105jrpyDvVlbV05cy4/m81WcX/pWnTo0CFp0qSJLF26VPr161d2/1/+8hdZuHChrFixosLzJ06cKJMmTTrtPNOnTzetMAAAwP3l5OTImDFjJD09XSIjI71n92RtedHxLOVbVJo1aybDhg075wutTtqbO3euDB06VIKCamerb09FXTmH+nIcdeU46so51Je1dWXvEXGEpUElJiZGAgIC5OjRoxXu19uNGjU67fkhISHmsLM3BuXm5rr8jabfGE18eu6ioiKXntvbUFfOob4cR105jrpyDvVlbV3puZQjnTqWBpXg4GDp0aOH/PjjjzJ69GhzX0lJibn9pz/96Zxfn5mZaS61VQUAAHgW/RyPiopy764f7coZN26c9OzZU3r37i2vvPKKZGdnm1lA5xIfHy/79++XiIgI8fPzc2m57N1Ken5Xdyt5G+rKOdSX46grx1FXzqG+rK0rbUnRkKKf4+dieVC58cYb5dixY/LUU0/JkSNHpFu3bjJnzhxp2LDhOb9WZwg1bdq0Rsun3xTexI6hrpxDfTmOunIcdeUc6su6ujpXS4rbBBWl3TyOdPUAAADfYvmCbwAAAGdCUDkDnV2ky/qXn2WEqlFXzqG+HEddOY66cg715Tl1ZemCbwAAAGdDiwoAAHBbBBUAAOC2CCoAAMBtEVQAAIDbIqhU4Y033pAWLVpIaGio9OnTR1auXCm+Tneu1tV/yx8dOnQoezwvL0/Gjx8v0dHRUrduXbn22mtP28PJmy1atEiuvPJKs8qi1s0XX3xR4XEds66LGjZu3Fjq1KkjQ4YMkZ07d1Z4zokTJ2Ts2LFmQaV69erJ7bffLllZWeJrdXXLLbec9l4bPny4T9bVCy+8IL169TKrb8fFxZmtRrZv317hOY787CUnJ8vIkSPNLvN6nkceecQr97dxpL4GDRp02vvrrrvu8rn6mjJlinTp0qVsEbd+/frJ7Nmz3fJ9RVCp5OOPPzbL+utUrLVr10rXrl3l8ssvl5SUFPF1nTt3lsOHD5cdixcvLnvsgQcekK+//lpmzpwpCxculEOHDsk111wjvkK3fdD3iobcqrz00kvy6quvytSpU2XFihUSHh5u3lf6y8BOP3i3bNlidin95ptvzAf6HXfcIb5WV0qDSfn32kcffVThcV+pK/1Z0g+L5cuXm9eqm8PpbvFah47+7BUXF5sPk4KCAlm6dKl88MEH8v7775vg7Iv1pf74xz9WeH/pz6ev1VfTpk3lxRdflDVr1sjq1avlsssuk1GjRpmfK7d7X+n0ZPyqd+/etvHjx5fdLi4utsXHx9teeOEFmy97+umnbV27dq3ysbS0NFtQUJBt5syZZfdt3bpVp73bli1bZvM1+rpnzZpVdrukpMTWqFEj28svv1yhzkJCQmwfffSRuf3LL7+Yr1u1alXZc2bPnm3z8/OzHTx40OYrdaXGjRtnGzVq1Bm/xlfrSqWkpJjXvnDhQod/9r777jubv7+/7ciRI2XPmTJlii0yMtKWn59v86X6UgMHDrTdd999Z/waX66v+vXr29555x23e1/RolKOJkNNl9osX34/Ib29bNky8XXaVaHN9a1atTJ/0Wqzn9I6079cytebdgs1b96cehORvXv3mn2syteP7nGh3Yr2+tFL7cLQzTnt9Pn6/tMWGF+zYMEC05Tcvn17ufvuu+X48eNlj/lyXaWnp5vLBg0aOPyzp5cXXHBBhf3TtDVPN5qz//XsK/VlN23aNImJiZHExESZMGGC5OTklD3mi/VVXFwsM2bMMC1P2gXkbu8rt9jrx12kpqaab1jlDRH19rZt28SX6YeqNuvpB4c2lU6aNEkGDBggmzdvNh/CwcHB5sOjcr3pY77OXgdVva/sj+mlfjCXFxgYaH7B+lodarePNjG3bNlSdu/eLY8//riMGDHC/GIMCAjw2boqKSmR+++/X/r3728+YJUjP3t6WdV7z/6YL9WXGjNmjCQkJJg/ujZu3CiPPvqoGcfy+eef+1x9bdq0yQQT7YLWcSizZs2STp06yfr1693qfUVQgUP0g8JOB2BpcNEf9k8++cQMDgVc5aabbiq7rn+x6futdevWppVl8ODB4qt07IX+YVB+bBicr6/yY5n0/aUD3PV9paFY32e+pH379iaUaMvTp59+KuPGjTPjUdwNXT/laFOg/sVWeWSz3m7UqJFl5XJHmrTbtWsnu3btMnWj3WZpaWkVnkO9lbLXwdneV3pZecC2jp7X2S2+Xofa1ag/m/pe89W60t3lddDw/PnzzSBIO0d+9vSyqvee/TFfqq+q6B9dqvz7y1fqKzg4WNq0aSM9evQwM6Z0kPu//vUvt3tfEVQqfdP0G/bjjz9WaD7U29o8hl/pVFD9C0T/GtE6CwoKqlBv2pSqY1ioNzFdGPqDW75+tB9Xx1PY60cv9ZeC9g3b/fTTT+b9Z/9F6qsOHDhgxqjoe83X6krHG+uHrjbJ62vU91J5jvzs6aU28ZcPdzojRqekajO/L9VXVbRFQZV/f/lKfVWmP0P5+fnu975y6dBcLzBjxgwzG+P99983swvuuOMOW7169SqMbPZFDz30kG3BggW2vXv32pYsWWIbMmSILSYmxoyqV3fddZetefPmtp9++sm2evVqW79+/czhKzIzM23r1q0zh/5YTZ482Vzft2+fefzFF18076Mvv/zStnHjRjOrpWXLlrbc3NyycwwfPtzWvXt324oVK2yLFy+2tW3b1nbzzTfbfKmu9LGHH37YzCzQ99q8efNsF154oamLvLw8n6uru+++2xYVFWV+9g4fPlx25OTklD3nXD97RUVFtsTERNuwYcNs69evt82ZM8cWGxtrmzBhgs3X6mvXrl22Z555xtSTvr/057FVq1a2Sy65xOfq67HHHjOzobQe9HeS3taZcz/88IPbva8IKlV47bXXzDcoODjYTFdevny5zdfdeOONtsaNG5s6adKkibmtP/R2+oF7zz33mOltYWFhtquvvtr8gvAV8+fPNx+6lQ+damufovzkk0/aGjZsaILw4MGDbdu3b69wjuPHj5sP27p165opfrfeeqv54PalutIPFP3Fp7/wdHpkQkKC7Y9//ONpfyj4Sl1VVU96vPfee0797CUlJdlGjBhhq1OnjvkDQ//wKCwstPlafSUnJ5tQ0qBBA/Nz2KZNG9sjjzxiS09P97n6uu2228zPl/5O1583/Z1kDynu9r7y039c20YDAADgGoxRAQAAbougAgAA3BZBBQAAuC2CCgAAcFsEFQAA4LYIKgAAwG0RVAAAgNsiqADwKn5+fvLFF19YXQwALkJQAeAyt9xyiwkKlY/hw4dbXTQAHirQ6gIA8C4aSt57770K94WEhFhWHgCejRYVAC6loUR3iy5/1K9f3zymrStTpkyRESNGSJ06daRVq1by6aefVvh63ZH1sssuM49HR0fLHXfcYXbrLu/dd9+Vzp07m/9Ld73VHXPLS01NlauvvlrCwsKkbdu28tVXX9XCKwdQEwgqAGrVk08+Kddee61s2LBBxo4dKzfddJNs3brVPJadnS2XX365CTarVq2SmTNnyrx58yoEEQ0648ePNwFGQ42GkDZt2lT4PyZNmiQ33HCDbNy4Ua644grz/5w4caLWXysAF3D5NocAfJbugBwQEGALDw+vcDz//PPmcf2Vo9vHl9enTx/b3Xffba6/9dZbZrfWrKyssse//fZbm7+/f9kOyvHx8bYnnnjijGXQ/+Ovf/1r2W09l943e/Zsl79eADWPMSoAXOrSSy81rR7lNWjQoOx6v379Kjymt9evX2+ua8tK165dJTw8vOzx/v37S0lJiWzfvt10HR06dEgGDx581jJ06dKl7LqeKzIyUlJSUs77tQGofQQVAC6lwaByV4yr6LgVRwQFBVW4rQFHww4Az8MYFQC1avny5afd7tixo7mulzp2Rceq2C1ZskT8/f2lffv2EhERIS1atJAff/yx1ssNwBq0qABwqfz8fDly5EiF+wIDAyUmJsZc1wGyPXv2lIsvvlimTZsmK1eulP/85z/mMR30+vTTT8u4ceNk4sSJcuzYMbn33nvld7/7nTRs2NA8R++/6667JC4uzsweyszMNGFGnwfA+xBUALjUnDlzzJTh8rQ1ZNu2bWUzcmbMmCH33HOPed5HH30knTp1Mo/pdOLvv/9e7rvvPunVq5e5rTOEJk+eXHYuDTF5eXnyz3/+Ux5++GETgK677rpafpUAaoufjqittf8NgE/TsSKzZs2S0aNHW10UAB6CMSoAAMBtEVQAAIDbYowKgFpDTzMAZ9GiAgAA3BZBBQAAuC2CCgAAcFsEFQAA4LYIKgAAwG0RVAAAgNsiqAAAALdFUAEAAG6LoAIAAMRd/T+HdtkVlwEcpAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "rng = jax.random.key(42)\n",
    "\n",
    "# Generate true data from y = w*x + b + noise\n",
    "true_w, true_b = 2, -1\n",
    "x_rng, noise_rng = jax.random.split(rng)\n",
    "xs = jax.random.normal(x_rng, (128, 1))\n",
    "noise = jax.random.normal(noise_rng, (128, 1)) * 0.5\n",
    "ys = xs * true_w + true_b + noise\n",
    "\n",
    "# Fit regression\n",
    "params = init(rng)\n",
    "losses = []\n",
    "for i in range(300):\n",
    "    loss_value, params = update(params, xs, ys)\n",
    "    losses.append(loss_value)\n",
    "\n",
    "plt.plot(losses)\n",
    "plt.xlabel(\"Epoch\")\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.title(\"Training Loss Over Time\")\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd2e8ea2",
   "metadata": {},
   "source": [
    "We’ve successfully trained the model with only pure functions!\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
